Methods for Evaluating Interactive Information Retrieval Systems with Users by Diane

Methods for Evaluating Interactive Information Retrieval Systems with Users by Diane Kelly is the first textbook to extensively focus on the methodologies of user studies in Information Retrieval (IR). This 224-pages book consists of 15 chapters covering a range of topics from the development of cognitive aspects in IR, to the foundation of research, to IR test collections, to the ethical issues of user studies. Chapter 1 opens the book by succinctly describing the problem in learning how to carry out a user study with interactive IR systems, that is, the experimental procedures are often not documented in the literature. Then Kelly reminds us that IR systems are fundamentally interactive and they should be evaluated from a user’s perspectives. Also, it was pointed out that IR has lacked the literature on the research methods and experimental designs when compared to other disciplines. This highlights the value of this book. Chapter 1 also contains a number of recommended readings on related topics in IIR and concludes with the outline of the book. In Chapter 2, Kelly presents a model to define Interactive Information Retrieval (IIR) using a continuum where one end has system focused studies (TREC-style studies) and the other end has human focused studies (Information-Seeking Behavior in Context). In between, several types of studies are placed along with the